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ABSTRACT 


While many institutions express interest in integrating sustainability into their civil engineer¬ 
ing curriculum, the engineering community lacks consensus on established methods for infus¬ 
ing sustainability into curriculum and verified approaches to assess engineers’ sustainability 
knowledge. This paper presents the development of a sustainability rubric and application of 
the rubric to civil engineering senior design capstone projects to evaluate students’ sustain¬ 
ability knowledge at two institutions. The rubric built upon previous assessment approaches to 
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evaluate student reports for nine different factors including dimensions of sustainability, Bloom’s 
taxonomy, sustainability links, drivers for including sustainability, location of sustainability within 
report, qualitative/quantitative incorporation, sustainability source/reference, and sustainability 
topics. The sustainability content within Spring 2014, Fall 2014, and Spring 2015 senior design 
capstone projects from university A (UA, n = 181 students, n p = 28 projects) and university B 
(UB, n = 106 students, n p = 15 projects) was evaluated using a mixed-methods approach. The 
mixed-methods assessment included observation of student project presentations and evalua¬ 
tion of student reports via rubric. Rubric evaluation of student reports revealed that students’ 
performance in senior design projects is primarily driven by their instructor’s expectations; if 
sustainability is not a major deliverable, then students are less likely to integrate sustainability 
concepts that they learned from prior classes in their reports. To make sustainability a priority, 
senior design project requirements should be updated to explicitly require holistic sustainabil¬ 
ity applications. Instructors could approach raising sustainability expectations by engaging a 
sustainability expert as an advisor to the senior design course and/or utilizing a sustainability 
expert as project mentor, as demonstrated in the success of one senior design project at each 
institution during this study. 

Key words: Sustainability, capstone design, mixed methods 


INTRODUCTION 


Engineers of the future must be prepared to address the complex, multidisciplinary problems 
that necessitate engineering solutions in sustainable and global contexts. Engineering education 
can provide students with the tools to approach these grand challenges of the 21 st century while 
considering aspects that are key for designing sustainable systems (David Allen et al. 2006, Davidson 
et al. 2010). Furthermore, according to the National Academy of Science report, Changing the 
Conversation, youth are seeking careers that make a difference (Sullivan 2011, National Academy of 
Sciences 2008). Sustainable engineering offers a solution to vital challenges, in conjunction with 
appealing to our youth (Oswald Beiler and Evans 2014). Furthermore, as of 2015 the Accreditation 
Board for Engineering and Technology (ABET) has recognized the importance of sustainability for 
student outcomes and in engineering curriculum; ABET criterion three and five have been updated 
to include engineering designs that meet desired needs within realistic constraints, such as sustain¬ 
ability, and curriculum that includes principles of sustainability (Accreditation Board for Engineering 
and Technology 2015). 
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LITERATURE REVIEW 


Strategies for assessing students’ sustainability knowledge and application are limited to a 
few studies (McCormick et al. 2014b, Bielefeldt 2013, Svanstrom, Lozano-Garcia, and Rowe 2008, 
Warburton 2003, Mckeown 2011, Riley, Grammes, and Thatcher 2007, Watson et al. 2013, McCormick 
et al. 2014a). The strategies include what topics need to be assessed and how to best measure 
student performance, including defining learning objectives related to assessing understanding of 
sustainable development via critical, holistic thinking and assessing the number of times a student 
mentions sustainably concepts, whether or not a student links three pillars of sustainability (envi¬ 
ronmental, economic, social), and utilizing instructor-created rubrics on course content or available 
assessments such as Sustainability in Higher Education Assessment Rubric (SHEAR) or Sustainability 
Assessment Survey (SAS) (Svanstrom, Lozano-Garcia, and Rowe 2008, Bielefeldt 2013, McCormick 
et ai. 2014b, Warburton 2003, Mckeown 2011, Vacca 2008). Despite the usability of instruments like 
SHEAR or SAS, assessments for sustainability in higher education often lack details indicative of 
interdisciplinary knowledge transfer necessary for learning about sustainability, thus researchers 
have recently adopted a concept mapping approach of assessing students. This approach com¬ 
pliments the nature of current global issues, which are complex and interconnected, and gauges 
whether students can rationally infer interactions between and within human and natural systems 
(Borrego et al. 2009). Conversely, the strategy for assessing student engineering knowledge and 
application is widely recognized in a culminating undergraduate engineering experience: senior 
design capstone projects (ASCE 2008). 

Incorporating sustainability into civil engineering curriculum presents considerable challenges 
to overcome the already constrained curriculum (Christ et al. 2014, Brown et al. 2014). Two main 
strategies have emerged from universities attempting to sustainability; termed herein as the stand¬ 
alone course method, and the module method. In the stand-alone course method, engineering 
programs establish one or two distinct, stand-alone courses that address sustainability and grand 
challenges in depth. Semester-long courses can enable an in-depth exploration of sustainability and 
sustainable engineering, enhancing students’ knowledge of both fundamentals and engineering 
applications for sustainability. In the module method, engineering programs integrate sustainability 
and grand challenges throughout a host of existing courses by threading individual sets of course 
skills together in an effort to reach higher levels of intellectual behavior via interdisciplinary con¬ 
cept connection (Fogarty 1991). Modules can be designed to fit into one lecture or over a series of 
lectures. Modules typically include everything an instructor needs for implementation: a summary 
of learning objectives and module activities, lecture slides and notes, recommended readings, and 
an assignment for students. 
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The current state-of-the-practice for senior design focuses on the design elements from the primary 
CEE sub-disciplines: Construction, Steel & Concrete Structures, Water Management and Infrastructure. 
The American Society for Civil Engineers (ASCE) Body of Knowledge 2 (BOK2) summarizes the required 
engineering content knowledge of twenty-four outcomes, organized into three types of outcomes: 
foundational, technical, and professional (ASCE 2008). Outcome ten is sustainability (ASCE 2008). 
The foundational outcomes create the base for continued learning in the technical and professional 
categories. Bloom’s taxonomy was adopted by BOK2 to define achievement goals of cognitive behavior, 
including “knowledge,” “comprehension,” “application,” “analysis,” "synthesis,” or “evaluation,” within the 
twenty-four outcomes (Anderson, Krathwohl, and Bloom 2001, Bloom et al. 1956). ASCE assigns each 
outcome a specific Bloom’s level; during the bachelor’s degree the expected level of achievement for 
the sustainability outcome is Bloom’s level knowledge, comprehension, and application. BOK2 expects 
that analysis be reached through work experience, i.e. after the bachelor’s degree. The sustainability 
outcome meets or exceeds Bloom’s taxonomy levels for sixteen of the twenty-four outcomes. BOK 
specifies the synthesis level for seven outcomes, including experiments, design, technical specialization, 
communication, lifelong learning, professional and ethical responsibility, and evaluation level for three 
outcomes, including design, technical specialization, professional and ethical responsibility (ASCE 2008). 

This paper develops a rubric for evaluating students’ sustainability knowledge based on integrat¬ 
ing previous assessment approaches with additional metrics. The rubric presented herein is applied 
to senior design capstone projects at two institutions employing stand-alone course method to 
evaluate students' use of sustainability within their designs. 


METHODS 


This methods section first provides an overview of the engineering curriculum at two U.S. institu¬ 
tions from their introduction to sustainability to their capstone experiences, University A (UA) and 
University B (UB). Next, the methods describe the development of a rubric for assessing how and to 
what cognitive extent students integrate sustainability concepts into senior design projects. And finally, 
the mixed-methods assessment is described, which includes observation of student senior design 
project presentations and rubric evaluation (Borrego, Douglas, and Amelink 2009). 

Engineering Curriculum and Senior Design Course Descriptions 

University A 

At University A (UA), sustainability is emphasized in teaching, learning, research, and operations. 
UA has made a significant investment in sustainability, including a sustainability minor for engineers, 
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which requires the completion of six sustainability courses. University A offers degrees in Civil and 
Construction Engineering and currently requires one stand-alone sustainable engineering under¬ 
graduate course, CEE 400 Earth Systems Engineering and Management (ESEM) that students take 
in their junior or senior year. University A also offers eight sustainable engineering elective courses 
that students may select to fulfill 18 elective credits required. University A represents the stand¬ 
alone course method; UA has established one distinct, required, stand-alone course, CEE 400 ESEM, 
which addresses sustainability and grand challenges in depth, and generally follows the content 
summarized in Allenby’s The Theory and Practice of Sustainable Engineering (Allenby 2014). 

University A requires all students to participate in the senior design course (CEE 486). The senior 
design project at UA encompasses a comprehensive land development plan involving engineering 
roles of due diligence, drainage, traffic circulation, water, wastewater, structural, and geotechnical 
analysis. Students work in teams of five to seven people per project and within each team students 
select a civil engineering sub-discipline role based on their interest. The teams are partnered with a 
local engineering firm whose role is to support students throughout their projects through mentor¬ 
ship and, in many cases, serve as ‘clients’ for student projects. The senior design project requires 
students to produce engineering design plans for their development, compile a comprehensive 
written report featuring all engineering sub-discipline roles, and present their engineering designs 
in a culminating presentation at the end of the semester. Sustainability is a required component of 
their engineering design; within each engineering subdiscipline, students are required to include 
innovative sustainability technologies as a stand-alone section that directly address design consid¬ 
erations within their project. A total of 181 students participated in UA senior design during Spring 
2014 (73), Fall 2014 (41), and Spring 2015 (67) semesters. 

University B 

University B’s (UB) Department of Civil and Environmental Engineering (CEE) has made a sig¬ 
nificant investment in sustainable engineering. While UB does not offer a minor in Sustainability, 
UB does have an Engineering for Humanity Certificate and (at the time of this writing) a nearly ap¬ 
proved University-wide Sustainability Certificate. Similar to UA, UB represents the stand-alone course 
method. Sustainable engineering faculty housed in CEE have developed and taught four stand-alone 
sustainable engineering undergraduate courses since 2008, including CEE 1209 Life Cycle Assess¬ 
ment (LCA) Methods and Tools, CEE 1210 Engineering and Sustainable Development (ESD), CEE 
1217 Green Building Design and Construction (GB), and CEE 1218 Design for the Environment (DFE). 

Students in CEE are required to take one of these four stand-alone courses that address sustain¬ 
ability and grand challenges in depth. CEE 1209 introduces students to LCA, including the methodol¬ 
ogy and tools used to conduct an LCA and follows Matthews, Hendrickson, and Matthews’ Life Cycle 


FALL 2017 


5 








ADVANCES IN ENGINEERING EDUCATION 

Utilizing Civil Engineering Senior Design Capstone Projects to Evaluate 
Students’ Sustainability Education across Engineering Curriculum 


Assessment: Quantitative Approaches for Decisions that Matter (Mathews, Chris T. Hendrickson, 
and Deanna H. Matthews 2015). CEE 1210 covers concepts of industrial ecology and sustainable 
development and follows Graedel and Allenby’s Industrial Ecology and Sustainable Engineering 
(Graedel and Allenby 2010). CEE 1217 introduces students to green buildings, life cycle of buildings, 
and utilizes the United States Green Building Council's Leadership in Energy and Environmental 
Design (LEED) green building rating system to demonstrate one possible green rating system (US 
Green Building Council 2007). CEE 1218 is a topical course that introduces students to concepts of 
design for environment tools and also includes in-depth investigations such as residential energy 
assessments. The Engineering and Sustainable Development (ESD) class and UA’s ESEM class are 
very similar in course content; the instructors are authors on this paper and collaborated in devel¬ 
oping the classes. More recently, UB’s Provost selects a theme to integrate throughout all curricula 
and activities; UB dedicated the 2014-2015 academic year to sustainability. 

The Civil and Environmental Engineering Department at UB requires all students to participate in 
the senior design course (CEE 1233/1333/1433/1533/1733/1833). Students work in teams of five to 
seven people per project. Each student takes on a civil engineering sub-discipline role within his or 
her larger senior design team. The teams are partnered with a local engineering firm or associate; 
the firm supports students throughout their projects through mentorship and exposure to 'real- 
world engineering’. The senior design project at UB encompasses a comprehensive engineering 
design simulated from real-world engineering projects. Students write a comprehensive report and 
present their project at the end of the semester. Sustainability is embedded in one section of the 
UB senior design project rubric; UB students are requested to consider constraints, one of which 
includes sustainability. A total of 106 students participated in UB senior design during Spring 2014 
(43), Fall 2014 (27), and Spring 2015 (36) semesters. 

Collection of Student Reports 

Students turned in their final senior design projects to their instructor on presentation day dur¬ 
ing the last week of the Spring 2014, Fall 2014, and Spring 2015 semesters. The authors collected 
the student reports after the completion of each course from the instructors. The same instructors 
taught senior design during the three semesters of this study; similarly the rubric and expectations 
given to the students remained the same during each semester at both universities. The projects 
and firms were different for every project team. 

Mixed-Methods Assessment 

The senior design projects were assessed using a mixed-methods approach combining observa¬ 
tion of student presentations and rubric evaluation of student reports. 
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Observation of Student Presentations 

Students in senior design courses at both UA and UB present their final projects to an audience 
of engineering professionals, their instructor and other engineering faculty on a single day at the 
end of the Spring 2014, Fall 2014, and Spring 2015 semesters. The authors, faculty and graduate 
students in Sustainable Engineering, viewed all senior design student presentations each semester 
and recorded their observations for sustainability content using a developed observation sheet. 
The observations of student presentations were, in part, used to develop and refine the rubric, 
which was then used to evaluate the sustainability aspects of the student projects. The observa¬ 
tion sheet contained the following five columns, which were used to guide notes taken during 
the presentation: 

1. Presentation title 

2. Sustainability Concepts Incorporated (Yes: please describe, or No) 

3. Was sustainability in the project client-driven, student-driven or other? (Client, Student, Rubric, 
or Other: please describe) 

4. Calculation or superficial incorporation of sustainability? (Calculation: please describe or 
Superficial: please describe) 

5. Source/reference cited for sustainability concept (Yes or No) 

The observers utilized Brundtland Commission definition of sustainability in this research; “meet¬ 
ing the needs of the present without compromising the ability of future generations to meet their 
own needs” (Commission 1987). Due to the number of students in each semester of senior design 
at UA, student presentations were split into two concurrent sessions, with one final presentation 
that everyone observed. The authors divided themselves between the rooms during the concurrent 
sessions and, to address consistency between author notes, two authors observed the final presen¬ 
tation together and compared notes afterwards. UB students presented their final projects in one 
session each semester; the authors were able to compare notes for each presentation. 

Development of Sustainability Rubric 

The rubric was developed to assess the sustainability content within students’ senior design 
projects. The rubric was developed in two phases; phase one derived best practices from a literature 
review of methods to assess sustainability content in student projects, and phase two developed 
new sustainability assessment measures and integrated them with best practices to create a holistic 
assessment tool (Mertler 2001). 

Phase one of rubric development mined best practices from literature approaches used to as¬ 
sess the sustainability content of student projects, which are summarized in the top half of the 
rubric described in Table 1. Bielefeldt 2013 utilized Dimensions of Sustainability to assess the pillars 


FALL 2017 


7 






ADVANCES IN ENGINEERING EDUCATION 

Utilizing Civil Engineering Senior Design Capstone Projects to Evaluate 
Students’ Sustainability Education across Engineering Curriculum 


Table 1. Sustainability rubric developed to assess student application of concepts. 


Criteria 


Possible Score 


1. Dimensions of Sustainability (Bielefeldt 
2013) 


Environmental 

Economic 

Social 


2. Cognitive levels of sustainability topics incorporated (Anderson 
et al. 2001) 


3. Sustainability Links (McCormick et al. 
2014) 


No Evidence 


Concepts 


No Evidence, Weak, Fair, Good 

1. Knowledge (recall of information) 

2. Comprehension (demonstrating, discussing) 

3. Application (applying knowledge, designing, 
experimenting) 

4. Analysis (recognizing trends and patterns) 

5. Synthesis (using old concepts to create new ideas) 

6. Evaluation (assessing theories and outcomes) 


Societal 

Economic 


Crosslinks 


Interdependent 


4. Was sustainability in the project client-driven, student-driven 
or other? 


5. Was sustainability integrated throughout report or stand-alone 
section of the report? 


6. Quantitative or qualitative incorporation of 
sustainability? 


Environmental 
Societal-Economic 
Economic-Environmental 
Environmental-Societal 
Societal-Economic-Environmental 
Student 
Client 
Other 

Rubric / Instructor 

Sustainability was integrated throughout sections 
Sustainability was stand-alone section in report 
Environmental Quantitative 
Economic 


Qualitative 


Social 


7. Source/ reference cited for sustainability concept 


Yes 

No 


8. Sustain- Sustainable Agriculture, Sustainable Land Use, Industrial Ecology, Corporate Sustainability, Climate 
ability Change, Renewable Energy, Green Buildings, Sustainability Infrastructure, Green Construction, LCA 
Topics (Life Cycle Assessment), Material Flow Analysis, Natural Resource Depletion (or Scarcity), Pollution 

(explicit/ Prevention, Design for the Environment, Green Chemistry, Environmental Justice, Embedded/Virtual 

implicit) Water Use, Anthropogenic Environmental Impacts, Sustainability Rating Schemes (e.g. LEED), Resilience, 
Urbanization/urban sprawl, Sustainability economics, Governance for sustainability. Sustainable Innovation, 
Sustainability Ethics, Other 1- recycling. Other 2- water reuse. Other 3- energy reduction. Other 4- Urban 
heat island effect. Other 5- alternative transportation, Other 6- consider needs of people/ stakeholder 
engagement, None 

Students’ senior design projects were evaluated via rubric to assess the sustainability content in the reports. The rubric 
included dimensions of sustainability (Bielefeldt 2013), Bloom’s taxonomy (Anderson et al. 2001), links (McCormick et al. 
2014), motivations, quantitative/qualitative incorporation of sustainability and references. Students had to score fair or good 
in dimensions of sustainability to be considered concept-level or greater in sustainability links. 
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of sustainability (environmental, economic, social) and the number of times (“no evidence” = no 
mention, “weak” = mentioned but no specific example, “fair” = mentioned one example, “good” = 
mentioned multiple examples) these concepts were incorporated into students’ projects (Bielefeldt 
2013). In addition, Bloom’s Taxonomy was utilized to assess levels of intellectual behavior within 
the student homework assignments (“knowledge,” “comprehension,” “application,” “analysis,” “syn¬ 
thesis,” or “evaluation”) (Anderson, Krathwohl, and Bloom 2001, Bloom et al. 1956). McCormick et 
al. 2014 utilized Sustainability Links to evaluate the linkages between the three pillars of sustain¬ 
ability, including “concepts” (societal, economic, environmental), “crosslinks” (societal-economic, 
environmental-economic, societal-environmental) and “interdependency” (societal-economic- 
environmental) (McCormick et al. 2014b). McCormick et al. 2014 did not include a “no evidence” 
response option; the authors added this option. Table 1 reflects these three approaches to assess 
Dimensions of Sustainability, Bloom’s Taxonomy, and Sustainability Links in student projects as 
criteria 1-3, respectively. 

Phase two of rubric development created additional sustainability assessment items based on 
the authors’ expertise and experience in sustainability. During observation of student presentations, 
the authors took notes on who seemed to drive the inclusion of sustainability, which was used to 
develop the rubric category, Drivers for Including Sustainability, which aims to gain insight into the 
motivating actors for incorporating sustainability into student report. In the rubric, drivers can include 
“student,” “client,” “other” and the combination of “rubric/instructor.” The rubric also documented 
where and to some extent how sustainability was integrated into student reports in the category, 
Location of Sustainability Within Report. Location assesses whether sustainability was "integrated 
throughout the report” or present in a “stand-alone section” only. The depth to which students ap¬ 
ply sustainability was added to the rubric in the category Quantitative/Qualitative Incorporation. 
This category evaluates whether sustainability was incorporated into the project via calculations 
and quantitative methods or superficial, qualitative methods for each of the three pillars of sustain¬ 
ability. Another rubric category for evaluating the depth to which students address sustainability 
reviews reports for references: the Sustainability Source/Reference category looks for sustainability 
citations. A list of sustainability topics, shown in Table 1, based on topics taught in students’ sustain¬ 
able engineering courses were used to create the rubric category Sustainability Topics. These topics 
were tracked as “implicitly presented” where students did not call out the topic directly but were 
discussing the topic, or “explicitly presented” where students directly described the topic in their 
report. Six “other” topics not covered in the stand-alone classes were added during the review of 
student projects based on common topics present in the student presentations, including recycling, 
water reuse, energy reduction, urban heat island effect, alternative transportation, consider needs 
of people/stakeholder engagement. 
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Bloom’s Taxonomy 

Bloom’s taxonomy provides syntax and a measurement scheme through which students’ levels of 
intellectual behavior can be assessed. Bloom’s taxonomy is divided into six compartments, including 
“knowledge,” “comprehension,” “application,” “analysis,” “synthesis,” and “evaluation” (Anderson, 
Krathwohl, and Bloom 2001). These levels were used within the rubric developed herein to create six 
Bloom’s cognitive levels. Projects were coded based on which of the six levels of Blooms students 
achieved; “knowledge” was coded if a student recalled a vocabulary term, “comprehension” was 
coded by discussion of vocabulary terms, “application” was coded by applying knowledge of vo¬ 
cabulary to design or problem-solve, “analysis” was coded by identification of patterns and trends, 
“synthesis” was coded by using old concepts to create new ideas, and “evaluation” was coded by 
comparing ideas or assessing theories. 

Dimensions of Sustainability 

Bielefeldt’s Dimensions of Sustainability were used to quantify the number of times a pillar of 
sustainability (economic, environmental, and social) was discussed in student reports, based on the 
Triple Bottom Line definition of sustainability (Elkington 2008). Students’ examples of dimensions of 
sustainability were judged on four criteria, including “no evidence” = no mention, “weak” = mentioned 
but no specific example, “fair” = mentioned one example, and “good” = mentioned multiple examples 
(Bielefeldt 2013). 

Sustainability Links 

McCormick’s et al. Sustainability Links were used to assess the connections and interrelatedness 
between the three dimensions of sustainability. Students’ examples of sustainability links were judged 
on three criteria, including “concept” = discussion of a topic(s) in relation to a single sustainability 
pillar, “crosslink” = discussion of a topic(s) in relation to two sustainability pillars, and "interdepen¬ 
dency” discussion of a topic(s) in relation to all three sustainability pillars (McCormick et al. 2014b). 

“Concepts” were defined as comprehension of sustainability topic in relation to a pillar of sustain¬ 
ability. As such, a project must demonstrate comprehensions by scoring “fair” or “good” in Dimensions 
of Sustainability to score concept-level in Sustainability Links. “Crosslinks” were defined as explicit 
or implicit discussion of two or more pillars of sustainability and their interaction. Students could 
achieve crosslinks through several avenues; students may explicitly describe the interconnectivity 
of two dimensions of sustainability in a single sentence or through several paragraphs implicitly 
link the two pillars of sustainability. “Interdependency” was defined as demonstrating knowledge of 
interconnectivity between the three dimensions of sustainability in the context of each project. As 
such, demonstration of interdependency necessitated demonstrating crosslinks, but demonstrating 
crosslinks did not always result in interdependency demonstration in all three pillars of sustainability. 
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Evaluation of Student Reports via Rubric 

Three graduate student evaluators utilized an Inter-Rater Reliability (IRR) approach to ensure that 
the evaluation and scoring of all 43 projects was consistent. IRR is defined as the process through 
which two or more raters classify subjects or objects independent of one another (Gwet 2014). High 
IRR verifies that the raters can be used interchangeably, thereby establishing the rater as an abstract 
entity to the main focus of study, the subjects (Gwet 2014, Armstrong et al. 1997). Utilizing an IRR 
approach, the rubric was applied to senior design projects in five steps; in step one the evaluators, 
scored one senior design project together, in step two the evaluators scored the same senior design 
project separately and met to discuss results, in step three the evaluators scored a different project 
and met to discuss results, and in step four the evaluators scored the rest of the projects and met 
to review all results. In step five, a fourth graduate student evaluator was utilized to score random 
senior design projects to ensure consistency amongst the previous three evaluators. 

UA and UB senior design reports from Spring 2014 and Fall 2014 were divided evenly between 
the three evaluators such that every person reviewed several projects from both of these semesters. 
During Spring 2015, the projects were gathered in paper format; evaluations were completed at 
each institution and not split among evaluators. 


RESULTS AND DISCUSSION 


Faculty and graduate students in Sustainable Engineering viewed all senior design student 
presentations each semester for binary (yes/no) presence of sustainability content based on the 
Brundtland Commission definition (Commission 1987). The observations revealed that all students 
at both UA and UB mentioned the incorporation of sustainability into their projects. However, binary 
approach to scoring student projects did not provide granularity needed to evaluate the spectrum of 
students’ incorporation of sustainability, indicating that further review of student projects via rubric 
analysis was necessary to differentiate between Triple Bottom Line sustainability (Elkington 2008). 

The rubric developed herein provides a method for evaluating student projects for knowledge of 
sustainability topics, level of cognitive use of sustainability, and students’ ability to apply sustain¬ 
ability at different depths, and students’ ability to use of quantitative and qualitative methods for 
sustainability. Results are discussed by each category of the rubric, presented in Figures 1-5. 

Dimensions of Sustainability 

Students’ senior design projects were marked on the Dimensions of Sustainability to understand 
students’ incorporation of environmental, economic, and social pillars of sustainability. Projects were 
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Dimensions of Sustainability Present in 
Students' Senior Design Projects 
University A and University B: S14, F14, S15 



Senior Design Projects (%) 

Figure 1. Dimensions of Sustainability Present in Senior Design Projects. 

University A (UA, n = 181 students, n p = 28 projects) and University B (UB, n = 106 students, 
n p = 15 projects) senior design projects were evaluated via rubric by three evaluators with 
expertise in sustainable engineering. The rubric evaluated the projects for Dimensions of 
Sustainability to understand students’ incorporation of environmental, economic, and social 
pillars of sustainability. Projects were assigned one of the following scores: “no evidence” = 
no mention, “weak” = mentioned but no specific example, “fair” = mentioned one example, or 
“good” = mentioned multiple examples based on the definitions provided (Bielefeldt 2013). 


assigned one of the following scores: “no evidence” = no mention, “weak” = mentioned but no specific 
example, “fair” = mentioned one example, or “good” = mentioned multiple examples based on the 
definitions provided (Bielefeldt 2013). Prior to the senior design course, all students at UA and UB had 
taken courses that covered all three pillars of sustainability. The results (Figure 1) show that the prevail¬ 
ing score of UA senior design projects achieved “good” in environmental pillar (79%), “fair” in economic 
pillar (54%), and “weak” in social pillar (32%). In comparison, the majority of UB senior design projects 
scored “fair” in social pillar (53%), “weak” in economic pillar (53%), and “weak” in environmental pillar 
(40%). UA and UB senior design projects most often discussed examples for one to two pillars; rarely did 
a single project discuss all three pillars of sustainability (environmental, social, and economic), despite 
exposure to all three pillars of sustainability within the required stand-alone sustainable engineering 
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course taken earlier in their educational careers. Senior-level students are expected to discuss multiple 
dimensions of sustainability reaching “fair” or “good” levels, according to ASCE BOK2. 

Students from UA performed strongest in environmental, followed by economic and social, while 
UB projects were strongest in economic, followed by environmental, and social. Students’ association 
of sustainability with environmental impacts more than other dimensions, combined by the ease of 
quantifying or connecting environmental issues, led to 100% of UA senior design projects discussing 
environmental aspects. Discussing economic issues in the context of sustainability was primarily 
done through cost comparison of different project scopes, which is a general rubric requirement for 
projects to provide cost analysis. UB projects performing stronger in economic than environmental 
may be due to the rigidity of project descriptions in addition to minimal incorporation of sustain¬ 
ability in the rubric. Lastly, social aspects of sustainability are weakest at both universities. While UB 
projects do a better job of incorporating social sustainability, it is difficult to measure students’ true 
understanding of social sustainability, because these projects inherently address social aspects (e.g. 
access to clean water, engaging stakeholders, or sustainability ethics) through sustainability-driven 
clients. While students discussed some social elements, it is possible they do not make the necessary 
connection between their project goals and societal implications. However, these outcomes are in 
line with hypothesized outcomes, where students show strong discussion in the environmental pillar 
followed by economic, and weak correlation in the social pillar. It is unknown why the two universities 
differ in strengths/weaknesses of incorporating the three pillars, but may be related to instructor 
emphasis within the senior design course and ease of addressing each pillar, where students tend 
to struggle most with social sustainability. 

Bloom’s Taxonomy 

Students’ senior design projects were scored based on Bloom’s Taxonomy to document students’ 
overall level of application of sustainability concepts. Projects were assigned one of the follow¬ 
ing scores: “knowledge,” “comprehension,” “application,” “analysis,” “synthesis,” or “evaluation” 
(Anderson, Krathwohl, and Bloom 2001, Bloom et al. 1956). The results (Figure 2) show that 57% of 
UA projects apply sustainability concepts at the “comprehension” level; these concepts were dem¬ 
onstrated through understanding of knowledge. In addition, 73% of UB projects apply sustainability 
concepts at the "knowledge” level; these concepts were demonstrated through recall of knowledge. 
The American Society for Civil Engineering (ASCE) Body of Knowledge 2 nd edition (BOK2) suggests 
that civil engineering students will reach up to Bloom’s level “application” for sustainability concepts 
by their senior undergraduate year (ASCE 2008). While 14% of UA and 13% of UB senior design 
projects reach “application” of sustainability and apply knowledge in new ways, the overwhelming 
majority of projects for both institutions do not reach this level. 


FALL 2017 


13 






ADVANCES IN ENGINEERING EDUCATION 

Utilizing Civil Engineering Senior Design Capstone Projects to Evaluate 
Students’ Sustainability Education across Engineering Curriculum 


Bloom’s Taxonomy Achieved in 
Students’ Senior Design Projects 
University A & University B: S14, F14, S15 


100% 



Knowledge Comprehension Application Analysis Synthesis Evaluation 


ASCE BOK2 
Outcome 10: 
Sustainability 


During Undergraduate 
Bachelor's Degree 


During Work 
Experience 


Undefined for 
Sustainability 
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Figure 2. Bloom’s Taxonomy Achieved in Senior Design Projects. 

University A (UA, n = 181 students, n p = 28 projects) and University B (UB, n = 106 students, 
n p = 15 projects) senior design projects were evaluated via rubric by three evaluators with 
expertise in sustainable engineering. The rubric evaluated the projects for Bloom's Taxonomy 
to understand students’ overall level of application of sustainability concepts. Projects were 
assigned one of the following scores: “knowledge,” “comprehension,” “application,” “analysis,” 
“synthesis,” or “evaluation” (Anderson, Krathwohl, and Bloom 2001, Bloom et al. 1956). 


Senior design projects are a culmination of students’ academic career, where they are asked to 
incorporate civil and environmental engineering learning into a multi-faceted project. Limited in¬ 
corporation of sustainability by instructors into the semester-long classroom furthered by minimal 
rubric weighting, leads project focus towards those rubric categories that benefit their final grade. 
Both stand-alone sustainability courses and sustainability modules should provide students with 
understanding of the relationship between sustainability and their core civil engineering curricu¬ 
lum, thereby enhancing students’ abilities to design, experiment, and analyze results (i.e. reaching 
“application”). This issue may be addressed by providing more examples of higher Bloom’s levels 
of sustainability applied to engineering design and/or requiring that students reach higher levels 
of sustainability application within their senior design projects through a combination of instructor 
request and course syllabus/project rubric requirements. 
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According to BOK2, students are expected to reach “analysis” during their post-undergraduate 
work experience. “Analysis” builds on modeling and experimentation through further identification, 
understanding, and interpretation of results. “Synthesis” and “evaluation” represent the highest 
levels of intellectual behavior and are typically associated with graduate-level study. These levels of 
cognition are not expected in senior design projects; “synthesis” would be demonstrated through 
the proposal of research, while “evaluation” would be demonstrated through comparing ideas and 
assessing theories. 

Sustainability Links 

Students’ senior design projects were evaluated on the linkages between the three pillars of 
sustainability, including “concepts” (societal, economic, environmental), “crosslinks” (societal- 
economic, environmental-economic, societal-environmental) and “interdependency” (societal- 
economic-environmental) (McCormick et al. 2014b). Because the definition for “concepts” was 
“recognition of the need to...” projects were required to first score “fair” or “good” in Dimensions of 
Sustainability, shown in Figure 1, to be considered “concept”-level or greater in Sustainability Links. 
As a consequence, projects with “no evidence” or "weak” evidence received "no evidence” scores 
for sustainability links, shown in Figure 3. UA results show 93% of projects displayed environmental 
concepts (100% projects showed environmental Dimensions of Sustainability however 7% of projects 
did not demonstrate recognition of environmental concepts), 61% of projects displayed economic 
concepts, and 39% of projects displayed social concepts related to sustainability. None of the UA 
senior design projects displayed interdependency between all three pillars of sustainability, however 
the most common (39%) cross-link was between environmental-economic. UB results show 60% of 
projects displayed social concepts, 27% of projects displayed environmental concepts, and 20% of 
projects displayed economic concepts related to sustainability. None of the UB senior design projects 
displayed interdependency between all three pillars of sustainability, and both environmental-social 
and economic-social pillars had the most linkages (13% each). No sustainability links were present 
in 7% of UA projects and 33% of UB projects. UA and UB students are exposed to the linkages be¬ 
tween sustainability pillars in their stand-alone sustainable engineering courses. At UA the required 
sustainability course (ESEM) that all students take covers all three pillars of sustainability; while the 
UB equivalent course (ESD) covers all three pillars, not all UB students take ESD. 

UA and UB students are exposed to the linkages between pillars in their sustainable engineer¬ 
ing courses. Students' deficiency in demonstrating these crosslinks further supports the idea that 
instructors and rubric drive students’ incorporation of sustainability into their project designs. 
Discussed in a later section, many projects incorporated sustainability into a separate section near 
the end of the project reports, containing minimal information, suggesting that sustainability was 
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Figure 3. Sustainability Links Present in Senior Design Projects. 

University A (UA, n = 181 students, n p = 28 projects) and University B (UB, n = 106 students, 
n p = 15 projects) projects were evaluated via rubric by three evaluators with expertise in 
sustainable engineering. The rubric evaluated the projects for Sustainability Links between the 
three pillars of sustainability. Projects were assigned one of the following scores: “concepts” 
(societal, economic, environmental), “crosslinks” (societal-economic, environmental- 
economic, societal-environmental) and “interdependency” (societal-economic-environmental) 
(McCormick et al. 2014b) and “no evidence”. Projects needed to score fair or good in 
Dimensions of Sustainability to be considered concept-level or greater in Sustainability Links. 


an afterthought, done only to meet rubric requirements. These findings suggest a need for deeper 
penetration of sustainability into instructor-student interaction time typically through class time, 
and succinct incorporation of sustainability requirements (i.e. quantitative and qualitative analysis) 
to the project rubric. 

Drivers for Including Sustainability 

The reason that students decided to include sustainability in their report was determined by 
reviewing the class rubric and students’ reports. The rubric category, Drivers for Including Sustain¬ 
ability, evaluates whether sustainability was integrated based on student interest (i.e. students dem¬ 
onstrated personal motivation towards sustainability), client request (i.e. client mission statement 
of students’ partnered firm addressed sustainability), rubric/instructor driven (i.e. rubric requires 
students to demonstrate underlying rationale behind incorporating sustainability and/or requires 
specific sustainability sections), or other (could also include projects that do not address sustain¬ 
ability). Projects could have multiple drivers, including student, client, or instructor. The findings 
show that all UA projects incorporated sustainability in the sections required by the senior design 
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rubric, which results in 100% of projects being rubric/instructor driven, while 21% of projects were 
student driven and 21% client driven. Conversely, only 40% of senior design projects from UB were 
determined to have rubric/instructor as the driver for sustainability. At UB, 33% of all projects 
were student driven, 33% were client driven, and 7% “other,” which documented non-sustainability 
drivers for the senior design project. The results show that the instructor and the rubric used in senior 
design has significant influence on drivers for incorporating sustainability within the senior design 
projects. UA’s rubric requires an explicit stand-alone section on sustainability; all student reports 
delivered on this requirement, though to different extents. Thus, reports showed higher instance of 
sustainability linkages. In comparison, UB’s rubric requires that students address sustainability, but 
it is embedded within the rubric under ‘addressing constraints’ for the overall project, not within 
each subdiscipline. 

Location of Sustainability Within Report 

Students’ senior design projects were evaluated for the Location of Sustainability Within Report 
as an indicator of the depth to which students apply sustainability beyond their standalone class. 
Senior design reports were scored based on how sustainability was integrated into the report, 
where the report either “integrated” sustainability throughout the report or “stand-alone” where 
sustainability was only in a single section of report. Reports could score either or both, depending 
on the location of sustainability. Sustainability was discussed in a stand-alone section for 100% of 
UA reports; 25% of UA reports discussed sustainability throughout the entire report in addition to 
discussions within the stand-alone sections. Conversely, 27% of UB reports discussed sustainability 
in a stand-alone section while 67% of reports discussed sustainability throughout the report. Seven 
percent of UB reports did not present any sustainability concepts in the report. Similar to Drivers 
for Including Sustainability, the project rubric has significant influence on location of sustainability 
within the senior design projects. All UA senior design project reports discussed sustainability in 
stand-alone sections as required by their rubric. In comparison, while required by the rubric to discuss 
constraints including, but not limited to, sustainability, 27% UB senior design projects discussed sus¬ 
tainability in stand-alone sections. As a result, sustainably was better woven throughout the project. 

Qualitative/Quantitative Incorporation 

Quantitative and qualitative incorporation of environmental, economic and social pillars of sus¬ 
tainably within students’ senior design projects are presented in Figure 4. Students’ senior design 
projects were scored for quantitative and qualitative incorporation of sustainability on a binary scale 
(0 = no evidence, 1 = evidence). The results show that all pillars of sustainability were incorporated 
qualitatively at both universities and that UA projects incorporated environmental and economic 
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Students' Senior Design Projects 
University A & University B: S14, F14, S15 
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Figure 4. Quantitative/Qualitative Incorporation of Sustainability in Senior Design 
Projects. 

University A (UA, n = 181 students, n p = 28 projects) and University B (UB, n = 106 students, 
n p = 15 projects) senior design projects were evaluated via rubric by three evaluators 
with expertise in sustainable engineering. The rubric evaluated the projects for type of 
incorporation for each sustainability pillar by tracking "quantitative” and “qualitative” within 
environmental, economic, and social. 


concepts quantitatively while UB projects incorporated only economic quantitatively. At both UA 
and UB the instructors do not require quantitative incorporation of sustainability but it is a strong 
recommendation. This finding suggests that students default to qualitative descriptions of sustain¬ 
ability rather than quantified metrics. The standalone sustainability classes that students are required 
to take include quantitative approaches to economic and environmental sustainability, so students 
should have these tools available to them. However, students may need additional examples of how 
to address sustainability through quantities in addition to qualities, thus greater emphasis should be 
placed on providing qualitative and quantitative applications of sustainability to senior design reports. 

Sustainability Source/Reference 

Students’ senior design projects were evaluated for their use of a citation, source, or reference 
as another method to evaluate the depth to which students apply sustainability. The Sustainability 


18 


FALL 2017 






































ADVANCES IN ENGINEERING EDUCATION 

Utilizing Civil Engineering Senior Design Capstone Projects to Evaluate 
Students’ Sustainability Education across Engineering Curriculum 



Source/Reference rubric category was scored with a binary “yes” or “no” for the presence of at least 
one reference supporting a sustainability statement or claim. Forty-three percent of UA students 
citied a reference for the sustainability concepts within their senior design projects; common sources 
include direct references for technologies, manufacturers and metrics for analyzing the sustainabil¬ 
ity of a product or process. No UB students cited a reference for the sustainability concepts within 
their senior design reports. In the standalone sustainability classes at both universities, students are 
taught how to find and use references within reports. However, the expectations for senior design 
differ from the stand-alone courses. A culture of citing sustainability sources should be fostered 
such that senior-level students understand the science behind sustainability. Rubrics should require 
sustainability citations in order for students to receive credit for discussing and connecting sustain¬ 
ability to their engineering designs. 

Sustainability Topics 

Finally, the rubric assessed the number and type of sustainability topics covered in an effort 
to understand concepts students applied within students’ chosen senior design projects. These 
topics were evaluated for the manner in which students included them; either explicitly or im¬ 
plicitly (Figure 5). Students’ senior design projects were scored for sustainability topics based 
on topics taught in the students’ Civil and Environmental Engineering (CEE) curriculum. The 
topics were tracked as “implicitly presented” where students did not mention the topic directly 
but discussed the topic, or “explicitly presented” where students directly mentioned the topic 
in their report. UA results show the greatest explicit incorporation of sustainable innovation 
(SUI), water reuse (WRE), and anthropogenic environmental impacts (AEI) while UA’s implicit 
incorporation of sustainability focused on sustainability infrastructure (SIF), pollution prevention 
(PPR), and renewable energy (REN). Conversely, UB’s greatest explicit incorporation of sustain¬ 
ability focused on stakeholder engagement (SEN), alternative transportation (ALT), and pollution 
prevention (PPR) while UB’s implicit incorporation of sustainability focused on sustainability 
infrastructure (SIF), pollution prevention (PPR), and sustainability economics (SEC). In CEE 400 
Earth Systems Engineering and Management, a required sustainable engineering course, UA civil 
engineering students are exposed to fifteen sustainability topics and despite this exposure none 
of the UA senior design projects incorporated climate change, pollution prevention, corporate 
sustainability, sustainability economics, sustainable agriculture, green buildings, and industrial 
ecology. Similarly, UB students are required to take one of the following three courses (additional 
may count toward elective credit): CEE 1209 Life Cycle Assessment Methods and Tools, CEE 1210 
Engineering and Sustainable Development, CEE 1217 Green Building Design and Construction, 
or CEE 1218 Design for Environment. Despite this, UB senior design projects incorporated topics 
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Sustainability Topics Present in Students’ Senior Design Reports 
University A & University B: S14, F14, S15 



Sustainability Topics 

AEI = Anthropogenic Envir Impacts 

ALT = Alternative Transportation 

CLC = Climate Change 

COS — Corporate Sustainability 

DFE = Design for the Environment 

EJU = Environmental Justice 

ERE = Energy Reduction 

GCH = Green Chemistry 

GCO = Green Construction 

GRB = Green Buildings 

GSU = Governance for Sustain. 

INE = Industrial Ecology 
LCA = Life Cycle Assessment 
MFA = Material Flow Analysis 
NRD = Natural Resource Depletion 
PPR = Pollution Prevention 
REC - Recycling 
REN = Renewable Energy 
RES = Resilience 

RSY = Rating Systems (e.g. LEED) 
SAG = Sustainable Agriculture 
SEC = Sustainability Economics 
SEN = Stakeholder Engagement 
SET = Sustainability Ethics 
SIF = Sustainability Infrastructure 
SLU = Sustainable Land Use 
SUI = Sustainable Innovation 
UHI = Urban Heat Island Effect 
UUS = Urbanization/Urban Sprawl 
VWU = Virtual Water Use 
WRE = Water Reuse 


Key 

Bold = Concepts Taught in Curriculum 
Red = Concepts NOT Mentioned 



Figure 5. Sustainability Topics Present in Senior Design Projects. 

University A (UA, n = 181 students, n p = 28 projects) and University B (UB, n = 106 students, 
n p = 15 projects) senior design projects were evaluated via rubric by three evaluators with 
expertise in sustainable engineering. The rubric evaluated the projects for Sustainability Topics 
incorporated into the students’ projects based on topics taught in the students’ Civil and 
Environmental Engineering (CEE) curriculum. The topics were tracked as “implicitly presented” 
where students did not mention topic directly but were discussing the topic, or “explicitly 
presented” where students directly mentioned the topic in their report. The topics were given 
a three-letter code. Bolded topic codes represent concepts taught in students’ curriculum. Red 
topic codes represent concepts not mentioned in students’ senior designs. Red bolded topics 
represent concepts taught to students but not mentioned in senior designs. 


of stakeholder engagement (SEN), alternative transportation (ALT), pollution prevention (PPR), 
sustainability economics (SEC), sustainability infrastructure (SIF), recycling (REC), sustain¬ 
able innovation (SUI), and green buildings (GRB). This finding suggests that despite extensive 
exposure to sustainability topics within their curriculum in a stand-alone sustainability class, 
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students do not apply these topics to their senior design projects. Students demonstrate the 
level at which their senior design rubric describes, and no more. Greater emphasis on higher 
cognitive application of sustainability and requirements to demonstrate knowledge of all three 
pillars may increase the number and level to which students integrate these sustainability topics 
within their senior design projects. 

For one project group at UA a sustainability expert served as the project mentor and client. The 
resulting senior design project achieved sustainability crosslinks, quantification for environmen¬ 
tal and economic sustainability, addressed 40% more sustainability topics than the next highest 
report (11 explicit and 4 implicit sustainability topics in this single report), and showed increased 
cognitive levels by achieving Bloom’s “application” level, per ASCE BOK2 requirement to profes¬ 
sional practice (ASCE 2008). This was a similar case for one project at UB, which also achieved 
“application” level Bloom’s, demonstrating that students can apply sustainability successfully to 
their senior design projects and suggesting that a sustainability expert may improve incorpora¬ 
tion of sustainability. 

During the evaluation of senior design projects for sustainability inclusion and cognition, the 
development of a holistic assessment tool synchronizing peer-reviewed evaluative methods was 
necessary. Independent application of the three peer-reviewed methods, Bloom’s Taxonomy, 
Dimensions of Sustainability and Sustainability Links, presents misleading results. Employing 
only Boom’s Taxonomy to assess senior design projects, in the context of sustainability, indicates 
an overwhelming majority of reports (86%) falling short of ASCE’s BOK2 cognitive achievement 
level, defined for senior undergraduates as the requirement for entry into the practice of civil 
engineering at the professional level. In addition, utilizing Bloom’s taxonomy independent of 
other peer-reviewed evaluative methods is insufficient to assess students’ demonstration of 
understanding the relationship between the three pillars of sustainability. Conversely, utilizing 
only Dimensions of Sustainability, the senior design projects score favorably; 69% of projects 
displayed “fair” or “good” evidence for environmental, 47% of projects “fair” or “good” in eco¬ 
nomic evidence, and 46% of projects “fair” or “good” social evidence. However, by definition this 
approach only provides a count of references from each pillar in a report and does not provide 
insight into the cognitive levels of student performance nor the demonstration of interconnected¬ 
ness between sustainability pillars. Due to the coupled assessment with Dimensions of Sustain¬ 
ability, applying only Sustainability Links results in few crosslinks (28% environmental-economic, 
19% environmental-social, and 7% societal-economic) in student projects. The holistic rubric 
presented herein covers a variety of aspects, including cognitive level, student understanding 
of topics and the linkages between topics, students’ ability to apply and calculate, and students’ 
use of sources to support their ideas. 


FALL 2017 


21 






ADVANCES IN ENGINEERING EDUCATION 

Utilizing Civil Engineering Senior Design Capstone Projects to Evaluate 
Students’ Sustainability Education across Engineering Curriculum 


FUTURE RUBRIC IMPLEMENTATION 


The authors found that future rubric implementation should include, at minimum, cognitive levels 
achieved, quantitative/qualitative, sustainability links, and sustainability topics. These four categories 
generated the greatest insight into students’ levels of cognition, quantification of sustainability pil¬ 
lars, demonstration of pillar linkages, and sustainability topics. Dimensions of sustainability provided 
redundant information; students’ ability to show more than one sustainability example was covered 
by aforementioned rubric sections. 


CONCLUSION 


Rubric evaluation of student reports revealed that students’ performance in senior design projects 
is primarily driven by their instructor’s expectations; if sustainability is not a major deliverable, then 
students are less likely to integrate sustainability concepts learned from prior classes in their reports. 
Despite this, when guided by project mentor and/or client with a sustainability mission, students 
demonstrated that they were capable of applying sustainability successfully to their senior design 
projects. The authors question whether students will translate their experiences in senior design to 
the civil engineering profession; if future clients do not request a high degree of sustainability, while 
there may be little-to-no incentive to incorporate it, these future civil engineers may not initiate it 
in on their own, as shown in the senior design results. 

In order to challenge students to draw upon the information learned throughout their previous 
classes, senior design project requirements should be updated to explicitly require holistic sus¬ 
tainability applications to the engineering designs. In addition, instructors could approach raising 
sustainability expectations by engaging a sustainability expert as an advisor to the senior design 
course and/or utilizing a sustainability expert as project mentor as demonstrated in one senior de¬ 
sign project. Not only would this approach support students throughout their senior design project 
but it would better prepare them for the role of a 21 st century engineer. 
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